Cache-Conscious Memory Management

Dynamic memory allocation must solve the re-allocation problem: where to place a new object. Current techniques of malloc and GC are designed with cache locality in mind. However, it is unclear how to cleanly separate the locality effect in these techniques. This paper outlines a locality theory for dynamic memory allocation. It presents a new metric called the reallocation distance, a way to compute the ideal locality, a linear-time algorithm to measure actual locality, and suggestions for using the theory in improving the design of memory allocators. 1. Reallocation Distance The heap space used to store an object is called a heap slot. The allocation events are of two types, allocation and free. In an execution, the events are interleaved in a linear sequence. The reallocation distance is defined for every allocation event. Let t be the time object o is allocated to heap slot A and t0 the time when A is last freed, i.e. the latest free before t. The reallocation distance is the number of heap slots involved in either allocation or free between t0 and t, including A. When a heap slot is used for the first time, it has no last free time. We say the reallocation distance is infinite. The new metric quantifies the effect of different memory allocations. Figure 1 shows two ways of allocating six objects in three heap slots and the reallocation distances for each allocation. The first way favors locality by always reusing the most recently freed heap slot. The effect is shown by the reallocation distance, which is shorter in the first way than in the second. Reallocation distance shows the locality of memory allocation. Specifically, the locality depends on the heap activity between the free of a heap slot and the subsequent reallocation of it. Good locality means as much heap reuse as possible, which means to minimize the number of heap slots involved, that is, to minimize the reallocation distance. 2. Optimal Allocation Locality We call the locality in memory allocation the allocation locality. The scientific (i.e. quantitative) definition of allocation locality is the reallocation distance. The two are synonymous in this paper. Furthermore, optimal allocation locality means the minimal reallocation distance. This section outlines an algorithm to compute the optimal locality. A